Noise Sensitivity of Static Neural Network Classifiers

A variety of artiicial neural networks are evaluated for their classiication abilities under noisy inputs. These networks include feedforward networks, localized basis function networks and exemplar classiiers. The performance of radial basis function classiiers deteriorate rapidly in presence of noise, but elliptical basis variants are able to adapt to extraneous input components quite robustly. For feedforward networks, selective pruning of weights based on an \optimal brain damage" approach helps in noise-tolerant classiication. Results from a radar classiication problem are presented. 1 Motivation ANN approaches to problems in the eld of pattern recognition and signal processing have led to the development of various \neural" classiiers using feed-forward networks 17, 18]. These include the Multi-Layer Perceptron (MLP) as well as kernel-based classiiers such as those employing Radial Basis Functions (RBFs) 1, 21]. A second group of neural-like schemes such as Learning Vector Quantization (LVQ) have also received considerable attention 12]. These are adaptive, exemplar-based classiiers that are closer in spirit to the classical K-nearest neighbor method. All these networks can serve as adaptive, non-parametric classiiers that learn through examples 17]. Thus, they do not require a good apriori mathematical model for the underlying signal characteristics. Moreover, a rmer theoretical understanding of the pattern recognition properties of feed-forward neural networks, that relates these properties to Bayesian decision making and to information theoretic results, has emerged recently 19, 20]. A good review of probabilistic, hyperplane, kernel and exemplar-based classiiers that discusses the relative merit of various schemes within each category, is available in 13, 17, 23]. Comparisons between these classiiers and conventional techniques such as decision trees, K nearest neighbor, Gaussian mixtures, and CART can be found in 23, 25]. It is seen that most of these networks show comparable performance over a wide variety of classiication problems, while providing a range of trade-oos in training time, coding complexity and memory requirements 9, 23]. Neural networks are not \magical". They do require that the set of examples used for training should come from the same (possibly unknown) distribution as the set used for testing the networks, in order to provide valid generalization and good performance on classifying unknown signals 4, 16]. Also, the number of training examples should be adequate and comparable to the number of eeective parameters in the neural network, for valid results 20, 22]. In this context, it is noted that cross-validation techniques can partially counter the eeects of small training …